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ABSTRACT 

The process used for evaluating computer software 
before it reaches the market may not be reliable or appropriate for 
the end user. School boards often develop their own evaluation 
procedures for teachers, but they may also be too technical, too 
long, or irrelevant to the teacher's purposes, and they may be too 
general for the discipline area in which the software will be used. A 
form appropriate for use by second language teachers should specify: 
lesson type; level (s) for which the materials are suitable; main 
topic; vocabulary and structures treated; whether accents can be 
input; appropriateness of cultural references; suitability of sound 
and graphics; quality of student instructions; quality of teacher 
documentation; feedback, to student; recordkeeping capabilities; and 
communicative uses suggested. The form should be brief and concise. 
In addition, teachers need to be trained in critical software 
evaluation that reflects targeted subject areas and teaching 
methodologies. A sample form is included here, with specific 
questions for the evaluator to ask about the materials and format. 
(Contains 11 references.) (MSE) 
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Teacher Evaluation of Foreign Language Software 



Janet Flewelling 

Janet Flewelling (Ed.D., Toronto) is an assistant professor of preservice 
*> and inservice French at the Faculty of Education, University of Windsor. 
Q Prior to this appointment, she taught Core French for the Etobicoke Board 
w of Education for 15 years. Her interests include second language 

methodology, teacher training and computers in education. 

The current evaluation process 



Software should go through a series of evaluations prior to being made 
available for purchase. In some cases, however, these evaluations can be 
superficial and unreliable. (Owston, 1986; Allesi and Trollip, 1985; Lillie, 
Hannum and Stuck, 1989). It is vital, therefore, that software destined for 
use in the school system be evaluated before it is catalogued for general 
use by teachers. Frequently boards of education will ask interested 
teachers to evaluate software they have purchased so that descriptions 
and comments about the usefulness of the software may be made available 
to other teachers. In addition, classroom teachers often preview, and 
therefore, at least subjectively evaluate software in order to make sure 
that it is suitable for their students. 

BEST COPY AVAILABLE 

Many boards of education have developed their own software evaluation 
_j forms for use by teachers, The best of these forms are concise and provide 
only the information that teachers really need to know in order to 
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determine whether software would be suitable for use by their students. 
Some boards, however, have made the mistake of patterning their 
evaluation form too closely after the forms used by professional 
evaluators. This results in teachers being faced with an evaluation scheme 
that can be too technical, too long and asking for information which is 
irrelevant to the classroom teacher's needs. 



What should teacher evaluation forms took like? 



A number of articles have addressed this question. (Bitter and Gore 1984; 
Alessi and Trollip, 1985; Lockard, Abrams and Many, 1987; Liliie, Hannum 
and Stuck, 1989). Flewelling (1989) made the following recommendations 
regarding the development of software evaluation forms for teacher use: 

-the form should not be too long 

-the form should not be cluttered 

-the print should be of such a size that it can be easily 

read 

-an overly complicated checklist should not be used. It 
should be sufficient to ask for yes/no or agree/disagree 
answers 

-use of jargon should be avoided 

-adequate room should be provided beside each question 

for comments 
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-questions which are subject specific should be included 
-questions should be divided into categories in order to 
make the form easier to follow 

Software evaluation and second language teachers 

As was previously stated, many boards of education provide teachers who 
wish to preview software with board-designed software evaluation forms. 
Typically, these forms are generalized so that software representing any 
subject area can be evaluated on that form. The question therefore arises: 
do these generalized forms meet the needs of second language (SL) 
teachers or would they benefit from a form designed to answer questions 
specifically related to SL study? The latter viewpoint seems to be 
advocated by the Joint Committee on Standards for Educational Evaluation 
(1981} which states that evaluation must serve the practical needs of a 
given audience. Lockard, Abrams and Many (1987) agree. They comment: 
"Our experience is that the most useful form for evaluating courseware is 
the one that speaks directly to the primary concerns of the evaluator (...) 
Therefore, we recommend that anyone setting out to evaluate courseware 
begin by developing a personal (...) form for that purpose". 

The logical outcome of this position is to develop an evaluation form 
specifically for SL teachers. In order to do so, one needs to draw up a list 
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of questions and criteria for SL software. SL teachers will need to know 
the following: 

-type of lesson 

-ievel(s) for which it is suitable 
-main topic 

-vocabulary and structures reflected 

-can accents be inputted? 

-are cultural references appropriate? 

-are sound effects and graphics suitable? 

-quality of the instructions to the student 

-quality of the teacher documentation 

-feedback provided for the student 

-does the program keep a record of student scores? 

-does the program suggest communicative uses? 

In developing an evaluation format for SL teachers, the challenge is to 
include the above information without making the form too long or too 
complicated. It is clear that a number of other questions need to be 
addressed. It would be hoped, however, that an initial evaluation of all 
software purchased would be done by the board's computer resource 
personnel in order to weed out any software unsuitable by the more 
generalized criteria of the board's standardized evaluation form. This 
would then allow SL teachers to concentrate on evaluating the software 



m ■ 

according to their more specialized criteria. 



Simply providing teachers with a form designed for the evaluation of SL 
software, however, is not sufficient. The majority of teachers have not 
been taught how to evaluate software. Unless they know what to look for 
and why, they will probably not be able to evaluate software in an 
informed and critical manner and their evaluations are likely to be less 
than reliable. A number of researchers stress the need for teachers to be 
trained to evaluate software (Ragsdale 1983; Caldwell 1983; Jones 1984). 
They should receive training on how to critically evaluate software and to 
recognize quality software. Moreover, the training should reflect the 
teaching methodologies associated with specific subject areas and grade 
levels. 

Many teachers, may feel that they do not have the time to take a course in 
software evaluation, or they may believe that since they are oily 
reviewing software in order to determine whether it is suitable for use in 
their own teaching program, a training program is not necessary- For these 
teachers it is imperative that, along with the software evaluation form, 
they be provided with a guideline on how to evaluate software. The 
guideline should be subject specific, short and concise so that the teacher 
will be less tempted to put it aside without reading it. 
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A proposed software evaluation form and guideline for SL 
teachers 

Based on the questions that researchers suggest need to be asked about SL 
software and software evaluation theory , the following evaluation form 
is suggested for use by SL teachers. It should always be accompanied by 
the software evaluation guideline included herein. 
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SECOND LANGUAGE SOFTWARE EVALUATION GUIDELINES 



The purpose of this guideline is to clarify some of the terms used in the 
evaluation form. You are encouraged tG read the guidelines before 
evaluating any SL software, When responding to questions, please use the 
back of the page if insufficient space is available on the front of the 
evaluation sheets. Ideally, your evaluation should reflect the reactions of 
students using the software as well as your own reactions to it. 



General Information 



Lesson type: is the program drill and practice, tutorial, simulation, 
adventure game or some other fo:~? (specify) 

Year of language study: since different school boards beg J n their 

SL programs at different grade 
levels, in addition to indicating the grade 
level(s) for which the program would be 
suitable, also indicate what year of SL study 
the grade(s) would represent. 

Main topic: indicate the central theme of the program 

Peripherals: is any equipment other than the computer required to run 
the program? (ie. printer, koala pad, joy stick, etc.) 

Description: briefly describe what the program allows the student to 
do. Use the back of the page, if necessary. 
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Content 



Vocabulary: 



Structures: 



Accents: 



the vocabulary used throughout the program should be 
neither too difficult nor too easy for the students. Any 
first/SL equivalents should be accurate. 

the structures used throughout the program should not be 
too difficult for the students. If the main theme of the 
program is a specific structure, indicate whether it is 
being taught or reviewed. 

it is highly important that SL software should allow 
students to add the required accents and diacritical 
marks to their work. Explanations of how to do this 
should be clear and the accents should look realistic. 



Culture: 



some software may deal specifically with a cultural 
theme. It should represent the culture accurately, in a 
modern light and without stereotyping. Culture may also 
be included incidentally and should adhere to the same 
criteria, 



Technical aspects 



Sound effects: 



sound effects can be fun but they can also overwhelm a 
program, becoming more important than the lesson 
itself. This is not desirable. Nor should they be too 
childish for the intended user age level. It is 
advantageous if the sound effects can be turned down or 
off, especially if the computer is to be used in the 
classroom while a lesson is being taught. 



Graphics: 



graphics as well can dominate a program to the extent 
that the students forget about the lesson at hand. They 
should add to the program without overwhelming it. 



Instructions: 



instructions should be clear enough to allow students to 
work independently. A help option should be available. 
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Documentation: documentation should provide information about 

the goals of the program, detail of how the 
program works, a complete description of what the 
program allows the student to do, an indication of 
what age/grade level the program is intended for, 
an explanation of how the program is intended to be 
used and what equipment is required to run the 
program, as well as any other details needed to 
enable the teacher to feel confident about using the 
program with students. 



Feedback: feedback should never be negative or insulting to the student. 
It should be age appropriate. A characteristic of a superior 
program is that not only will it inform students of when they 
have made a mistake, but it will also indicate what the 
mistake is. 



Scoring: It is desirable that the program indicates to the students their 
level of performance through some sort of scoring mechanism. 
Ideally, these scores could be retained for teacher reference. 

Overall: the program should adhere to good SL teaching pedagogy. It 
should enhance student learning, not just entertain. It should 
be fun for the students. It should not crash through an 
inadvertent mistake on the user's part. Preferably, it should 
integrate with some aspect of the classroom curriculum and 
should encourage in some way communication in and/or about 
the target language. Software which allows teachers to input 
their own information can be particularly useful since this 
facilitates the integration of the program with the curriculum 
materials being used in class. 



Comments: provide your reaction to the program. Would you 

recommend it to other teachers? How would you suggest 
that it be incorporated into the classroom program? What 
communicative activities could it generate? Use the back of 
the page if necessary. 
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SECOND LANGUAGE SOFTWARE EVALUATION FORM 



Teacher: 



School: 



General Information 

Program name: 

Language: 



Lesson type (Please check): 



Grade level: 

Main topic(s): 

Make of computer: 

Peripherals: -Printer 

- Koala pad 

- Joy stick 
Brief description: 



Catalogue number # 



-Drill and practice 
-Tutorial 
-Simulation 
•Game 

- Other (Please specify) 



Year of language study: 



Content 

Vocabulary - does it reflect a theme? 

- are the equivalents accurate? 

- is it suitable for the grade level? 

Structures - is a particular structure stressed? 

- is it suitable for the grade level? 

Accents - do accents need to be created? 

- can they be created easily? How? 

• are the instructions clear on how to make them? 
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Culture: 



- is it presented in an accurate and up to date manner? 

- is it free of stereotypes? 



Technical aspects 



Sound effects: 

Graphics: 
instructions: 



Documentation: 
Feedback: 

Scoring: 

Overall: 



Summary 

Comments: 



are they an asset or irritating? 
can they be turned down or off? 

do they add or detract? 



are they clear? 

can the student work independently? 
are help options available? 



is it adequate? 



is it positive and free from insult? 
is error analysis provided? w 



does the program keep student scores? _ 
can they be stored for teacher rt ieval? 

is the program pedagogically sound? 

does it enhance learning? 

is it fun for the students? 

is it crash proof? 



does it integrate with your program? 
can you input your own information? 
does it encourage communication? 
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Recommendations 

The use of computers in SL programs is still in its infancy. Their potential 
as motivators, tools and tutors, however, is high. Research indicates that 
computer-assisted instruction (CAI) can have a very positive effect on 
student motivation and learning. (Patterson and Smith, 1986; Lockhard, 
Abrams, and Many, 1987). But CAI can only be as effective as the software 
used in the process. It therefore is important that SL teachers make every 
attempt to find good quality software which can be used to enhance their 
teaching program and facilitate learning on the part of their students. Use 
of the software evaluation form and guideline proposed in this paper may 
facilitate this task. 
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